Toward a New Protocol to Evaluate Recommender Systems
نویسندگان
چکیده
In this paper, we propose an approach to analyze the performance and the added value of automatic recommender systems in an industrial context. We show that recommender systems are multifaceted and can be organized around 4 structuring functions: help users to decide, help users to compare, help users to discover, help users to explore. A global off line protocol is then proposed to evaluate recommender systems. This protocol is based on the definition of appropriate evaluation measures for each aforementioned function. The evaluation protocol is discussed from the perspective of the usefulness and trust of the recommendation. A new measure called Average Measure of Impact is introduced. This measure evaluates the impact of the personalized recommendation. We experiment with two classical methods, K-Nearest Neighbors (KNN) and Matrix Factorization (MF), using the well known dataset: Netflix. A segmentation of both users and items is proposed to finely analyze where the algorithms perform well or badly. We show that the performance is strongly dependent on the segments and that there is no clear correlation between the RMSE and the quality of the recommendation.
منابع مشابه
A New WordNet Enriched Content-Collaborative Recommender System
The recommender systems are models that are to predict the potential interests of users among a number of items. These systems are widespread and they have many applications in real-world. These systems are generally based on one of two structural types: collaborative filtering and content filtering. There are some systems which are based on both of them. These systems are named hybrid recommen...
متن کاملEvaluation of recommender systems: A multi-criteria decision making approach
The evaluation and selection of recommender systems is a difficult decision making process. This difficulty is partially due to the large diversity of published evaluation criteria in addition to lack of standardized methods of evaluation. As such, a systematic methodology is needed that explicitly considers multiple, possibly conflicting metrics and assists decision makers to evaluate and find...
متن کاملAn Effective Algorithm in a Recommender System Based on a Combination of Imperialist Competitive and Firey Algorithms
With the rapid expansion of the information on the Internet, recommender systems play an important role in terms of trade and research. Recommender systems try to guess the user's way of thinking, using the in-formation of user's behavior or similar users and their views, to discover and then propose a product which is the most appropriate and closest product of user's interest. In the past dec...
متن کاملContext-Aware Recommender Systems: A Review of the Structure Research
Recommender systems are a branch of retrieval systems and information matching, which through identifying the interests and requires of the user, help the users achieve the desired information or service through a massive selection of choices. In recent years, the recommender systems apply describing information in the terms of the user, such as location, time, and task, in order to produce re...
متن کاملMerging Similarity and Trust Based Social Networks to Enhance the Accuracy of Trust-Aware Recommender Systems
In recent years, collaborative filtering (CF) methods are important and widely accepted techniques are available for recommender systems. One of these techniques is user based that produces useful recommendations based on the similarity by the ratings of likeminded users. However, these systems suffer from several inherent shortcomings such as data sparsity and cold start problems. With the dev...
متن کامل